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IMAGE PROCESSING APPARATUS AND METHOD FOR BINARIZING 
A MULTILEVEL IMAGE 

Background of the Invention 
5 Field of the Invention 

The present invention relates to a 
multilevel image processing technology. 

Description of the Related Art 

10 Recently, a slip recognition technology using 

a non-contact type image input device, such as 
an over-head reader (OHR) , has become a key for 
winning financial OCR (optical character reader) 
business. 

15 An OHR is a stand type image input device 

provided with a line or area CCD (charge couple diode) 
as an image element, as shown in Fig. 1A. Compared with 
a conventional contact type image input device, such 
as an image scanner, etc., by using an OHR, entry to 

20 a slip can be made possible while a user is inputting an 
image and an image can be inputted while viewing a list 
of slips. Therefore, work can be performed comfortably. 

Compared with an image obtained by a 
scanner (hereinafter called "a scanner image"), an 

25 image obtained by the OHR (hereinafter called "an 
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OHR image") suffers from degradation, such as that 
caused by uneven gradation, reflection, image 
distortion, etc. 

Fig. IB shows an example of a scanner image, 
5 and Fig. 1C shows an example of an OHR image. The 
OHR image shown in Fig. 1C does not include 
the reflections of desks, walls, human beings, etc., 
and it is of fairly good quality for an OHR 
image. However, compared with the scanner image shown 
10 in Fig. IB, the OHR image has a large degree of 
uneven gradation and character lines that are more 
blurred. If an OHR is used, there is also a case where 
an OHR image with reflections, as shown in Fig . ID must 
be handled, since there is a possibility that 
15 the reflections of desks, walls, human beings, etc., 
may be included in an image. The OHR image shown in 
Fig. ID is blurred from the right to the left of the 
image due to reflections and as if the image were 
gradated. If an OHR is used, the development of a 
20 base technology for overcoming such image 
degradation becomes a major problem. 

In order to configure a high-precision 
binarizing system for an OHR image, it is necessary to 
obtain a character outline which is resistant 
25 against reflection and uneven gradation. Therefore, 
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constant threshold value binarization is not 
sufficient and Niblack's local binarization (see 
Reference 1: IEEE Transactions on Pattern Analysis and 
Machine Intelligence, Vol. 17, No. 12, p. 1191-1202, 
5 1995), etc., must be introduced. 

Niblack T s local binarization is a system 
of performing binarization for each pixel assuming 
that the threshold value of each pixel T = E + K o 
(E: average gray level of pixels in the vicinity of 

10 a target pixel, a : standard deviation of gray level 
of the pixels in the vicinity of the target pixel, 
K: prescribed constant) . A rectangular area of N X N 
(N is a constant) with the target pixel located at 
the center is used as the vicinity of the target pixel. 

15 However, if a conventional system, such 

as Niblack's binarization, etc., is used 
without modification, a black-white flickering noise 
occurs since all pixels in the vicinity of the pixel 
have an even gray level inside a background or a thick 

20 line. 

Fig. IF shows a binary image obtained 
by performing Niblack's local binarization (N=7, 
K=-0.1) for the OHR image shown in Fig. IE. According 
to the conventional binarization system, a black- 
25 white flickering noise occurs, as shown in Fig. IF. Such 
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a noise in which "black" and "white" are flickering 
must be eliminated. 

Although a method of eliminating the black- 
white flickering noise which occurs in the case 
5 where Niblack's local binarization is applied is 
described in the previous reference, the method is 
complex, the process requires many steps and the 
calculation cost is high, which is a problem. 

10 Summary of the Invention 

It is an object of the present invention 
to eliminate a black-white flickering noise, which is 
the problem of the conventional binarization system, 
at a cost that is as low as possible. 

15 In order to solve the problem described above, 

a background judgment device judging whether each 
target pixel is a background pixel based on 
information obtained in the vicinity of the target pixel, 
is introduced in the present invention, and is used 

20 in combination with a local binarization, such 
as Niblack's, etc. 

One aspect of the present invention comprises 
a background judgment device and a local 
binarization device. On receipt of a multilevel image, 

25 the background judgment device judges for each 
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pixel whether the pixel is a background pixel. If it 
is judged that the pixel is not a background pixel, 
the local binarization device locally binarizes the 
pixel, judges whether the pixel belongs to a background 
5 or a stroke composing a character, ruled line, etc., 
and outputs a binary image. 

Another aspect of the present invention 
comprises a background judgment device, a local 
binarization device and a line element restriction 

10 device. On receipt of a multilevel image, the background 
judgment device judges whether the pixel is a background 
pixel. If it is judged that the pixel is not a 
background pixel, the local binarization device 
locally binarizes the pixel, judges whether the pixel 

15 belongs to a background or a stroke composing a 
character, ruled line, etc., and outputs a binary image . 
If the ratio of the black pixels in a shape-fixed line 
element mask, including a target pixel of the obtained 
binary image is a prescribed value or more, the line 

20 element restriction device leaves the black pixels as 
black pixels or converts all pixels in the line element 
mask into black pixels. 

Another aspect of the present invention 
comprises a background judgment device, a local 

25 binarization device, a line element restriction device 
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and a stroke separation device. On receipt of a 
multilevel image, the background judgment device 
judges for each pixel whether the pixel is a background 
pixel . If it is judged that the pixel is not a background 
5 pixel, the local binarization device locally binarizes 
the pixel, judges whether the pixel belongs to a 
background or a stroke and outputs a binary image. If 
the ratio of the black pixels in the shape-fixed line 
element mask, including a target pixel of the obtained 

10 binary image is a prescribed value or more, the line 
element restriction device leaves the black pixels as 
black pixels or converts all pixels in the line element 
mask into black pixels. Then, the stroke separation 
device applies binarization to the partial pattern in 

15 the gray scale image, corresponding to the black 
pixel joint element of the obtained binary image, 
and divides the pattern into strokes of different 
gray levels. 

Since in any of the configurations 

20 described above, the background judgment device 
roughly judges whether a target pixel is a background 
pixel prior to the local binarization, the occurrence 
of a black- white flickering noise can be suppressed 
at low calculation cost. 

25 
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Brief Descriptions of the Drawings 

Fig. 1A shows the appearance of an over- 
head reader (OHR) . 

Fig. IB shows an example of a scanner image. 
5 Fig. 1C shows an example of an OHR image without a 

shadow. 

Fig. ID shows an example of an OHR image with 
a shadow. 

Fig. IE shows an example of an OHR image. 
10 Fig. IF shows a binary image obtained by 

applying Niblack's local binarization to the OHR image 
shown in Fig. IE. 

Fig. 2A shows the configuration of an 
image processing apparatus in the first embodiment of 
15 the present invention. 

Fig. 2B shows the configuration of the 
image processing apparatus in the second embodiment of 
the present invention. 

Fig. 3 shows an area in the vicinity of a 
20 target pixel. 

Fig. 4 shows a binary image obtained by 
applying the process of the second embodiment of the 
present invention to the OHR image shown in Fig. IE. 

Fig. 5 shows the configuration of the 
25 image processing apparatus in the third embodiment of 



the present invention. 

Fig. 6 shows an average gray level difference. 
Fig. 7 shows a binary image obtained by 
applying the process of the second embodiment of the 
5 present invention to the OHR image shown in Fig. ID. 

Fig. 8 shows a binary image obtained by 
applying the process of the third embodiment of the 
present invention to the OHR image shown in Fig. ID. 

Fig. 9 shows the configuration of the 
10 image processing apparatus in the fourth embodiment of 
the present invention. 

Fig. 10 shows "r", which is used in the 
fourth embodiment of the present invention. 

Fig. 11 shows the relationship between "r", 
15 which is used in the fourth embodiment of the 
present invention, and a black pixel ratio t. 

Fig. 12 shows the configuration of the 
image processing apparatus in the fifth embodiment of 
the present invention. 
20 Fig. 13 shows another OHR image. 

Fig. 14 shows a binary image obtained by 
locally binarizing the OHR image shown in Fig. 13. 

Fig. 15 shows a binary image obtained by 
applying the process of the fifth embodiment of the 
25 present invention to the OHR image shown in Fig. 13. 
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Fig. 16 shows the configuration of the 
image processing apparatus in the sixth embodiment of 
the present invention. 

Fig. 17 shows an example of a slant mask. 
5 Fig. 18 shows another OHR image. 

Fig. 19 shows a binary image obtained by 
applying the process of the third embodiment of the 
present invention to the OHR image shown in Fig. 18. 

Fig. 2 0 shows a binary image obtained by 
10 applying Niblack's local binarization to the OHR image 
shown in Fig. 18. 

Fig. 21 shows a binary image obtained by 
applying the process of the sixth embodiment of the 
present invention to the OHR image shown in Fig. 18. 
15 Fig. 22 shows the configuration of the 

image processing apparatus in the seventh embodiment 
of the present invention. 

Fig. 23 shows the basic process of a 
stroke separation device. 
20 Fig. 24 shows an information processing 

device, which is used to configure the image 
processing apparatus of the present invention. 

Fig. 25 shows how to provide the software 
program, etc., of the present invention. 



25 
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Description of the Preferred Embodiments 

The embodiments of the present invention 
are described with reference to the drawings. 

Fig. 2A shows the configuration of the 
5 image processing apparatus in the first embodiment of 
the present invention. This apparatus comprises 
a background judgment device 101 and a 
local binarization device 102. On receipt of a 
multilevel image, the background judgment device 101 

10 judges for each pixel whether the pixcel is a background 
pixel. If it is judged that the pixel is not a 
background pixel, the local binarization device 102 
locally binarizes the pixel, judges whether the pixcel 
belongs to a background or a stroke and outputs a 

15 binary image. 

Fig. 2B shows the configuration of the 
image processing apparatus in the second embodiment of 
the present invention. This apparatus comprises 
a background device by standard deviation 201 and 

20 a local binarization device 202. On receipt of 
a multilevel image, the background judgment device 
by standard deviation 201 judges for each pixel 
whether the pixel is a background pixel . If it is judged 
that the pixel is not a background pixel, the 

25 local binarization device 102 locally binarizes the 
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pixel, judges whether the pixel belongs to a background 
pixel or stroke and outputs a binary image. 

The background judgment device by 
standard deviation 201 judges whether a target pixel 
5 is a background pixel, using the standard deviation o 
of the gray level of pixels in the vicinity area of 
the target pixel. Specifically, if a < a min 
(predetermined constant) , it is considered that the 
gray level in the vicinity of the target pixel is even 

10 and it is judged that the pixel is a background pixel 
if this condition is met. The vicinity area of a target 
pixel is the vicinity of the pixel which is a joint area 
including the target pixel. An example of this is shown 
in Fig. 3. In Fig. 3, a rectangular area of N X N with 

15 a target pixel 301 located at the center (N = 7 in Fig. 3) 
is the pixel vicinity area 302 . The shape of the vicinity 
area is not limited to a rectangle and a circle, diamond, 
etc., are also available. It is permissible if the 
target 301 is not located at the center of the vicinity. 

20 The standard deviation of the gray level of the pixels 
in the vicinity area 302 is assigned as the standard 
deviation of the target pixel 301. 

The local binarization device 202 

performs Niblack's local binarization for only the 

25 pixels which the background judgment device 201 judges 
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not to be a background pixel. Niblack's binarization 
is a binarization method using an amount which 
is calculated by T = E + K a using both the average of 
the gray level E and the standard deviation o in 
5 the vicinity area of a target pixel. 

Fig. IF shows a result obtained by 
applying Niblack's local binarization to the OHR image 
shown in Fig. IE, and Fig. 4 shows a result obtained 
by applying the process of the image processing 

10 apparatus in the second embodiment of the present 
invention to the image. If the results of Fig. IF and 
Fig. 4 are compared, it is found from the process result 
of the second embodiment, in which Niblack's 
local binarization is performed after the 

15 background judgment by standard deviation, that most 
of the black-white flickering noises are eliminated. 
In the process of obtaining the result shown in Fig. 
4, a rectangular area of 7 X 7 with a target pixel 
located at the center is used as a vicinity area, and 

20 it is assumed that amin = 10 and K = -0.1. 

Fig. 5 shows the configuration of the 
image processing apparatus in the third embodiment of 
the present invention. This apparatus comprises 
a background judgment device by average gray 

25 level difference 501 and a local binarization device 
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502. On receipt of a multilevel image, the 
background judgment device by average gray level 
difference 501 judges for each pixel whether the pixel 
is a background pixel. If it is judged that the pixel 
5 is not a background pixel, the local binarization 
device 502 locally binarizes the pixel, judges whether 
the pixel belongs to a background or a stroke and 
outputs a binary image. 

The background judgment device by average 
10 gray level difference 501 judges whether a target pixel 
is a background pixel, using an average gray 
level difference A g in the vicinity area of the 
target pixel. 

Average gray level difference Ag is an amount 
15 that is defined by the following equation. 

A g = Average gray level of white pixels in 
the vicinity area - Average gray level of black pixels 
in the vicinity area. 

The average gray level of white pixels in 
20 the vicinity area and the average gray level of 
black pixels in the vicinity area are temporarily 
determined by a temporary binarization threshold value 
T T . 

Average gray level difference Ag is described 
25 with reference to Fig. 6. First, in the vicinity 601 
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of a target pixel m, pixels a through jS and pixels 
A through U are temporarily determined to be 
white pixels and black pixels, respectively, by 
threshold T 1 - Then, the average gray level Gl of white 
5 pixels a through J3 and the average gray level G2 of 
black pixels A through U are calculated, and average 
gray level difference A g is calculated from the 
difference (G1-G2) . The calculated Ag is assigned as 
the average gray level difference Ag of the target pixel 
10 m. 

If there are both a background area and a 
stroke area in the vicinity area, the average gray 
level difference Ag becomes great and if there is either 
a background area or a stroke area, the average 

15 gray level difference becomes small. Therefore, if in 
the background judgment device by average gray 
level difference 501, average gray level difference A 
g < Agmin (predetermined constant) , it is judged that 
a target pixel is a background pixel. Then, the 

20 local binarization 502 applies Niblack's local 
binarization only to pixels which the background 
judgment device by average gray level difference 501 
judges not to be background pixels. 

Fig. 7 shows a result obtained by applying 

25 the process of the image processing apparatus in 
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the second embodiment of the present invention, 
which comprises a background judgment device by 
standard deviation, to the OHR image shown in Fig. ID, 
and Fig. 8 shows a result obtained by applying the 
5 process of the image processing apparatus in the third 
embodiment of the present invention, which comprises 
a background judgment device by average gray level 
difference, to the OHR image shown in Fig. ID. In the 
processes of obtaining the results shown in Figs. 7 and 

10 8, a rectangular area of 7 X 7 with a target pixel 
located at the center is used as the vicinity area and 
it is assumed that amin =10, K = -0.1 and Agmin = 8. 

Since Fig. ID shows an OHR image with a 
shadow, there is a possibility that a background may 

15 also be judged as a black pixel in a shadowed part, and 
the gray level difference between a stroke and a 
background becomes small, Therefore, as shown in Fig . 7, 
strokes cannot be sufficiently extracted and 
are blurred in the image processing apparatus, 

20 which comprises a background judgment device by 
standard deviation, while as shown in Fig. 8, strokes 
can be extracted without blur and a good binarization 
result with little noise can be obtained in the 
image processing apparatus, which comprises a 

25 background judgment device by average gray level 
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difference . 

Although the background judgment device by 
average gray level difference 501 in the third 
embodiment of the present invention judges that a target 
5 pixel is a background pixel if average gray level 
difference Ag < Agmin (predetermined constant) , the 
average gray level difference also becomes small if 
there are only stroke areas in the vicinity area. 
Therefore, a pixel which should naturally be recognized 
10 to be a stroke is sometimes judged as being a background 
pixel. For example, although the characters of "ff-[S| 
10" shown in Fig. ID, which is an original image, are 
black, the characters of "it 159 1 0" shown in Fig. 8, 
which is the process result in the third embodiment of 
15 the present invention, are white-punched. In order to 
prevent this, a process of further judgment about 
whether the target is a black or white pixel, based on 
the gray level of the target pixel is added after the 
judgment by the background judgment device by average 
20 gray level difference 501 in the third embodiment of 
the present invention about whether the target pixel 
is a background pixel as described earlier. In this 
way, strokes can be prevented from being white-punched. 

Fig. 9 shows the configuration of the 
25 image processing apparatus in the fourth embodiment of 
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the present invention. This apparatus comprises 
a background judgment device by "r" 901 and a 
local binarization device 902. On receipt of a 
multilevel image, the background judgment device by "r" 
5 901 judges for each pixel whether the pixel is 
a background pixel, and if the pixel is not a 
background pixel, the local binarization device 902 
locally binarizes the pixel, judges whether the pixel 
belongs to a background or a stroke and outputs a 

10 binary image. 

If it is assumed that the standard deviation of 
the gray level of pixels in the vicinity of a 
target pixel and the gray level difference in the even 
area of the target pixel are cr and Ag, respectively, 

15 "r" in the background judgment device by "r" 901 is 
calculated by the following equation, 
r = a / Ag 

The background judgment device by "r" 901 judges whether 
the target pixel is a background pixel, using the 

20 calculated "r". 

Here, "r" is described with reference to Figs. 
10 and 11. Fig. 10 is an example of a specific 
multilevel image and shows a target pixel 1001, a 
vicinity area 1002, a black area in the vicinity area 

25 1003, a white area in the vicinity area 1004, a stroke 
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1005 and a background 1006. 

It is assumed that the average gray level of 
the black area in the vicinity 1003 and the average 
gray level of the white area in the vicinity area 1004 
5 are gl and g2, respectively. Then, the following 
equation holds true. 

Standard deviation o = rX|gl - g2 1 = rAg 
Specifically, r = a/Ag. If the black pixel ratio t 
in the vicinity area is assigned, "r" is represented 
10 by the following equation (1), and the "r" and 
black pixel ratio t have the relationship shown in Fig. 
11. 



r = f(t) 

15 = (t(l-t)) 1/2 

= (t-t 2 ) 1/2 

= (1/4- (t-1/2) 2 ) 1/z (1) 



Therefore, if "r" is small, the black pixel 
20 ratio t can be small. If the black pixel ratio t is 
small, specifically, a black area is narrow or there 
is very little black area, and the target pixel can be 
judged to be a background pixel. Therefore, if r < 
rmin (predetermined constant) is satisfied, the 
25 target pixel is judged to be a background pixel. Since 
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r is a quadratic function, r and t are not determined 
one-to-one. Accordingly, there is a possibility that 
even if r < rmin is satisfied, the target pixel is not 
a background pixel. However, this can be handled by 
5 the background judgment device by average gray 
level difference in the third embodiment of the 
present invention executing the same process as that 
executed to prevent strokes from being white-punched. 

The local binarization device 902 

10 applies Niblack's local binarization to only pixels 
which the background judgment device of "r" 901 judges 
not to be a background pixel. 

Fig. 12 shows the configuration of the 
image processing apparatus in the fifth embodiment of 

15 the present invention. This apparatus comprises 

a background judgment device 1201, a local 
binarization device 1202 and a line element restriction 
device (No. 1) 1203. The feature of this embodiment is 
that the line element restriction device (No. 1) 1203 

20 eliminates noises which cannot be formed into a 
line element since a stroke is composed of line 
elements, which is formed by joining pixels, and the 
accuracy of background judgment can be improved. 

In the image processing apparatus shown in 

25 Fig. 12, on receipt of a multilevel image, the 
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background judgment device 1201 judges for each pixel 
whether the pixel is a background pixel, and if the pixel 
is not a background pixel , the local binarization device 
1202 locally binarizes the pixel, judges whether the 

5 pixel belongs to a background or a stroke and outputs 
a binary image. If all pixels in a shape-fixed 
line element mask including the target pixel in 
the obtained binary image are black, the line 
element restriction device 1203 leaves all the pixels 

10 in the line element mask as black, and if they are not 
so, it modifies all the pixels in the line element 
mask to white pixels. 

Here it is assumed that the line element mask 
used in the line element restriction device (No. 1) 

15 1203 is a rectangular mask with a horizontal/vertical 
ratio of 1 X 3 or 3 X 1 . Since generally a stroke 
is composed of three dots or more, a mask of 1 X 3 or 
3X1 can be used and noises which do not amount to 
the thickness of a stroke can be eliminated. In this 

20 way, a stroke on a checkered pattern where one dot of 
a black pixel and one dot of a white pixel 
are alternately arrayed can be extracted. The 
checkered pattern where one dot of a black pixel and 
one dot of a white pixel are alternately arrayed is 

25 recognized as a gray-painted area by human eyes and this 
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is a technology which an image processing apparatus 
which receives slips as input images requires. 

Fig. 13 shows the OHR image of a character 
"00230" in a slip, which is written on a checkered 
5 pattern which is recognized as a gray-painted area. Fig. 
14 shows a result obtained by applying a 
local binarization to the OHR image shown in Fig. 13, 
and Fig. 15 shows a result obtained by applying 
the process of the image processing apparatus in the 

10 fifth embodiment of the present invention to the OHR 
image shown in Fig. 13. In Fig. 14, there are 
black-white flickering noises in a background area, 
while in Fig. 15 there are very little black-white 
flickering noise and clear strokes are extracted. 

15 Fig. 16 shows the configuration of the 

image processing apparatus in the sixth embodiment of 
the present invention. This apparatus comprises 
a background judgment device 1601, a local 
binarization device 1602 and a line element restriction 

20 device (No. 2) 1 603 . Although the configuration of this 
embodiment (Fig. 16) is almost the same as that of the 
fifth embodiment (Fig. 12), they differ in the 
detailed processes of the line element restriction 
devices . 

25 In the image processing apparatus shown in Fig. 16, 
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on receipt of a multilevel image, the 
background judgment device 1601 judges for each pixel 
whether the pixel is a background pixel . If it is judged 
that the pixel is not a background pixel, the 

5 local binarization device 1602 locally binarizes the 
pixel, judges whether the pixel belongs to a background 
or a stroke and outputs a binary image. If the ratio 
of the black pixels in the shape-fixed line element 
mask including the target pixel in the obtained 

10 binary image is a prescribed value or more, the black 
pixels are left as black pixels or all pixels in the 
line element mask are converted into black pixels. If 
the ratio of the black pixels in the line element mask 
is less than the prescribed value, all pixels in the 

15 line element mask are converted into white pixels. 

For the line element masks used in the 
line element restriction device (No. 2) 1603, there are 
a total of six kinds of masks: four kinds of 
rectangular masks with horizonal/vertical ratios of 13 

20 X 1, 1 X 13, 5X3 and 3X5, with a target pixel 
located at the center, and two kinds of slant masks of 
5X3 and 3X5. Fig. 17 shows a slant mask of 5 X 3 
1701. The line element restriction device (No. 2) 1603 
converts all pixels in the line element mask if 11 pixels 

25 are black pixels in the line element mask, and, 
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otherwise, it converts all pixels in the line element 
mask into white pixels. In this way, good strokes can 
even be extracted from blurred strokes. 

Although the image processing apparatus 
5 is configured in such a way that the background 
judgment device 1601 judges, the local binarization 
device 1602 performs a local binarization and the line 
element restriction device 1603 eliminates black- 
white flickering noises from the obtained binary image, 
10 it can also be configured in such a way that the 
local binarization device 1602 performs a local 
binarization without the background judgment by the 
background judgment device 1601 and the line element 
restriction device 1603 eliminates the black-white 
15 flickering noises from the obtained binary image. 

Fig. 18 shows an example of an OHR image with 
a shadow. The ruled lines in the lower right section 
of Fig. 18 are blurred. Fig. 19 shows a binary 
image obtained by applying the process of the 
20 third embodiment of the present invention, in 
which background judgment by average gray level 
difference is made and a local binarization is performed, 
to the image shown in Fig. 18. In this case, 
strokes composing the ruled lines in the lower right 
25 section are blurred. Fig. 20 shows a binary image 
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obtained by applying Niblack's local binarization to 
the image shown in Fig. 18 without any background 
judgment. In this case, although there are black-white 
flickering noises in the background area of Fig. 20, 

5 it must be noted that strokes composing the ruled lines 
are clearly extracted. Furthermore, Fig. 21 shows a 
binary image obtained by applying the process of the 
sixth embodiment of the present invention to the image 
shown in Fig. 18. Although Fig. 21 includes noises, 

10 strokes composing the ruled lines are clearly extracted. 
There is a possibility that good ruled lines may 
be extracted by eliminating lines short of a 
prescribed length based on the length restriction of 
ruled lines, etc., in a subsequent stage. 

15 Fig. 22 shows the configuration of the 

image processing apparatus in the seventh embodiment 
of the present invention. This apparatus comprises 
a background judgment device 2201, a local 
binarization device 2202, a line element restriction 

20 device 2203 and a stroke separation device 2204. The 
feature of this embodiment is that the stroke separation 
device 2204 separates two strokes of different gray 
levels if they touch. According to this 
embodiment, characters can be accurately extracted 

25 from a gray scale image in which a ruled line and a 
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character touch. 

On receipt of a multilevel image, the 
background judgment device 2201 judges for each pixel 
whether the pixel is a background pixel . If it is judged 
5 that the pixel is not a background pixel, the 
local binarization device 2202 locally binarizes the 
pixel, judges whether the pixel belongs to a background 
or a stroke and outputs an binary image. If the ratio 
of the black pixels in the shape-fixed line element 
10 mask including a target pixel of the obtained binary 
image is a prescribed value or more, the line 
element restriction device 2203 leaves the black pixels 
as black pixels or converts all pixels in the 
line element mask into black pixels. Then, the 
15 stroke separation device 2204 calculates the black 
pixel joint elements of- the binary image obtained by 
the line element restriction device and applies 
Otsu's binarization (see Reference 2: Technical Report 
of The Institute of Electronics, Information 
20 and Communication Engineers '80/4, Vol. J63-D, No. 
4, p. 349-356, 1980) to a partial pattern in a gray 
scale image corresponding to each joint element. If 
an inter-class dispersion is a prescribed value or 
more or a dispersion ratio (intra-class 
25 dispersion/inter-class dispersion) is less than a 
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prescribed value, the partial pattern is divided into 
two strokes. 

Fig. 23 shows the concept of the process of 
the stroke separation device 2204. Otsu's binarization 
5 is applied to the partial pattern 2301 obtained by 
the line element restriction device 2203. Since 
the partial pattern 2301 is composed of two strokes 
of different gray levels (a character stroke 2303 and 
a ruled line stroke 2302), the inter-class 

10 dispersion becomes a fairly large value. If the 
calculated inter- class dispersion is a prescribed 
value or more, the partial pattern 2301 is divided into 
two strokes. If the partial pattern is composed of 
strokes of a small gray level difference, which is not 

15 shown in Fig. 23, the stroke separation device 2204 
judges that they are the same kind of stroke and the 
pattern is not divided. 

If the stroke separation device 2204 uses 
a dispersion ratio instead of the inter- 

20 class dispersion, it is configured to divide the 
partial pattern 2301 into two strokes when the 
dispersion ratio is less than a predetermined value. 

The image processing apparatuses described 
above can be configured using the information 

25 processing device (computer) shown in Fig. 24. The 
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information processing device shown in Fig. 24 
comprises a CPU (central processing device) 2401, a 
memory 2402, an input device 2403, an output device 2404, 
an external storage device 2405, a medium driving device 
2406 and a network connection device 2407, and they 
are connected to one another by a bus 2408. 

The memory 2402 includes, for example, a 
ROM (read-only memory) , a RAM (random access 
memory) , etc., and stores a program and data which are 
used for the process. The CPU 2401 executes necessary 
processes by using the memory 2402 and running the 
program. Specifically, the background judgment, 
local binarization, line element restriction and 
stroke separation described in each embodiment of the 
present invention are realized by the program stored 
in the memory 2402. 

Image data, such as a slip, etc., are entered 
into the information processing device via the input 
device 2403, such as an OHR, etc. The output device 
2404 includes, for example, a display, printer, etc., 
and is used to output process results, etc. 

The external storage device 2405 includes, 
for example, a magnetic disk device, an optical 
disk device, a magneto-optical disk device, etc. 
The information processing device can store the 
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program and data described above in this external 
storage device and can use the program and data by 
loading them into the memory 2402, if required. 

The medium driving device 2406 drives the 
5 portable storage medium 2409. For the portable storage 
medium 2409, an arbitrary computer-readable storage 
medium, such as a memory card, a floppy disk, a CD- 
ROM (compact disk read-only memory) , an optical disk, 
a magneto-optical disk, etc., is used. The program 
10 and data described above are stored in this 
portable storage medium 2409 and can be used by loading 
them into the memory 2402, if required. 

The network connection device 2407 
communicates with an external device via an arbitrary 
15 network (line), such as a LAN (local area network) , etc., 
and performs a data conversion 

accompanying communications. The information 

processing device can receive the program and data 
described above from the external device via the network 
20 connection device 2407 and can use the program and data 
by loading them into the memory 2402, if required. 
Although Fig. 24 shows a single information processing 
device, the system can also be implemented by a 
processing device consisting of a plurality of 
25 computers or a plurality of processing devices via a 
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network. 

Fig. 25 shows how to provide a software 
program, etc., to be run by the information processing 
device of the present invention. For example, such a 
5 program, etc., can be provided by one of the following 
three methods. 

(a) Such a program, etc., is installed 
and provided in the information processing device, 
such as a computer, etc. In this case, such a 

10 program, etc., are, for example, pre-installed at a 
plant before shipment. 

(b) Such a program, etc., is stored and 
provided in the portable storage medium 2502. In this 
case, the program, etc., stored in the portable storage 

15 medium 2502 is installed in the external storage device 
2405 of the information processing device 2501, such 
as a computer, etc. 

(c) Such a program, etc., is provided from 
a server in the network 2503. In this case, 

20 the information processing device 2501, such as 
a computer, etc., usually obtains the program, etc., 
by downloading the program, etc., stored in the 
server 2504. 

In this case, the server 2504 generates a 

25 signal for transmitting a program, etc., and transmits 
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the signal to the information processing device 2501 
via an arbitrary transmission medium in the network 
2503. 

In this way, according to the present 

5 invention, if image data, such as a slip, etc., which 
are obtained from an input device, such as an OHR, 
black- white flickering noises, which are 
conventionally generated in a background area, can be 
eliminated, and thereby high-accuracy binarization can 

10 be realized. 

One aspect of the present invention comprises 
a background judgment device and a local 
binarization device. On receipt of a multilevel image, 
the background judgment device judges for each 

15 pixel whether the pixel is a background pixel. If it 
is judged that the pixel is not a background pixel, 
the local binarization device locally binarizes the 
pixel, judges whether the pixel belongs to a background 
or a stroke composing a character or ruled line 

20 and outputs a binary image. In this way, since 
the background judgment device roughly judges whether 
a target pixel is a background pixel, the occurrence 
of black-white flickering noises can be suppressed. 

Another aspect of the present invention 

25 comprises a background judgment device, a local 
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binarization device and a line element restriction 
device. On receipt of a multilevel image , the background 
judgment device judges for each pixel whether the pixel 
is a background pixel. If it is judged that the pixel 
5 is not a background pixel, the local binarization 
device locally binarizes the pixel, judges whether the 
pixel belongs to a background or a stroke and outputs 
a binary image. If the ratio of black pixels in 
the shape-fixed line element mask including a target 

10 pixel in the obtained binary image is a prescribed value 
or more, the line element restriction device leaves 
the black pixels as black pixels or converts all 
pixels in the line element mask into black pixels. In 
this way, the line element restriction device can 

15 eliminate noises short of a line element and the 
accuracy of background judgment can be improved. 

Another aspect of the present invention 
comprises a background judgment device, a local 
binarization device, a line element restriction device 

20 and a stroke separation device. On receipt of a 
multilevel image, the background judgment device 
judges for each pixel whether the pixel is a background 
pixel. If the pixel is not a background pixel, the local 
binarization device performs a local binarization, 

25 judges whether the pixel belongs to a background or a 
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stroke and outputs a binary image. If it is judged that 
the ratio of the black pixels in the shape-fixed line 
element mask including a target pixel in the obtained 
binary image is a prescribed value or more, the line 
5 element restriction device leaves the black pixels as 
black pixels or converts all pixels in the line element 
mask into black pixels. Then, the stroke separation 
device applies binarization to the" partial pattern in 
a gray scale image corresponding to the black pixel 
10 joint element and divides the partial pattern into 
strokes of different gray levels. In this way, the 
stroke separation device can extract high-accuracy 
characters from a gray scale image in which a plurality 
of strokes, such as a ruled line, character, etc., 
15 touch. 
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What is claimed is : 

1. An image processing apparatus, comprising: 

a background judgment device judging whether 
5 a target pixel is a background pixel using a 
standard deviation of gray level of pixels in a vicinity 
area of the target pixel on receipt of a multilevel 
image . 

10 2 . An image processing apparatus, comprising: 

a background judgment device judging whether 
a target pixel is a background pixel using a gray 
level difference and a standard deviation of gray levels 
of pixels in a vicinity area of the target pixel 

15 on receipt of a multilevel image. 

3. The apparatus according to claim 2, wherein 

the gray level difference is an amount which 
is calculated based on a difference between an 
20 average gray level of white pixels in the vicinity area 
of the target pixel and an average gray level of black 
pixels in the vicinity area of the target pixel. 

4. An image processing apparatus, comprising: 

25 a background judgment device judging for 
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each target pixel whether the target pixel is a 
background pixel on receipt of a multilevel image; and 
a local binarization device locally binarizing 
the target pixel, judging which of a background and 
5 a stroke the target pixel belongs to, and outputting 
a binary image if it is judged that the target pixel 
is not the background pixel. 

5. The apparatus according to claim 4, wherein 

10 said local binarization device uses an 

amount which is calculated based on an average and a 
standard deviation of gray levels of pixels in the 
vicinity area of the target pixel as a binarization 
threshold for the target pixel. 

15 

6. The apparatus according to claim 5, wherein 

the amount which is calculated based on 
the average and the standard deviation of the gray 
levels of the pixels in the vicinity area of the target 
20 pixel is calculated based on a sum of the average and 
a constant-multiple of the standard deviation. 

7. The apparatus according to claim 5, wherein 

the vicinity area of the target pixel is 
25 a rectangular area of N X N with a prescribed number 
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of pixels N and the target pixel located at a center. 

8. The apparatus according to claim 4, wherein 

said background judgment device judges whether 
5 the target pixel is the background pixel, using a 
standard deviation of gray levels of pixels in the 
vicinity area of the target pixel. 

9. The apparatus according to claim 8, wherein 

10 said background judgment device judges whether 

the target pixel is the background pixel under 
a background judgment condition of a < o min with o 
as the standard deviation in the vicinity area of 
the target pixel and a min as a prescribed constant. 

15 

10. The apparatus according to claim 4, wherein 

said background judgment device judges whether 
the target pixel is the background pixel using a 
standard deviation of gray levels and a gray level 
20 difference of pixels in the vicinity area of the target 
pixel . 

11. The apparatus according to claim 10, wherein 

said background judgment device judges whether 
25 the target pixel is the background pixel under 
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a background judgment condition of r = a/ Ag< rmin with 
a as the standard deviation in the vicinity area of the 
target pixel, Ag as the gray level difference in the 
vicinity of the target pixel and rmin as a prescribed 
5 constant. 

12. The apparatus according to claim 10, wherein 

said background judgment device judges whether 
the target pixel is the background pixel under 
10 a background judgment condition of Ag < Agmin with A 
g as the gray level difference in the vicinity of 
the target pixel and Agmin as a prescribed constant. 

13. The apparatus according to claim 10, wherein 

15 the gray level difference is an amount which 

is calculated based on a difference between an 
average gray level of white pixels in the vicinity area 
of the target pixel and an average gray level of black 
pixels in the vicinity area of the target pixel. 

20 

14. The apparatus according to claim 10, wherein 

said background judgment device judges whether 
the target pixel is a background pixel using a 
combination of a background judgment conditions o < o 
25 min, r = a/Ag < rmin and Ag < Agmin with a as the 
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standard deviation in the vicinity area of the target 
pixel, Ag as the gray level difference in the vicinity 
of the target pixel and omin, rmin and A grain as a 
prescribed constant. 

15. The apparatus according to claim 4, 
further comprising: 

a line element restriction device executing 
a process of the obtained binary image based on a 
ratio of black pixels in a shape-fixed line element 
mask including the target pixel and outputting a 
binary image. 

16. The apparatus according to claim 15, wherein 

said line element restriction device leaves 
the black pixels in the line element mask as black 
pixels if the ratio of black pixels in the line element 
mask is a prescribed ratio or more. 

17. The apparatus according to claim 15, wherein 

said line element restriction device converts 
all pixels in the line element mask into black pixels 
if the ratio of black pixels in the line element mask 
is a prescribed ratio or more. 
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18. The apparatus according to claim 15, wherein 

said line element restriction device converts 
all pixels in the line element mask into white pixels 
if the ratio of black pixels in the line element mask 
is less than a prescribed ratio. 

19. The apparatus according to claim 15, wherein 

said line element restriction device uses 
a plurality of line element masks. 

20. The apparatus according to claim 4, 
further comprising: 

a stroke separation device applying a 
partial pattern in a gray scale image corresponding to 
a black pixel j oint element in the obtained binary image 
and separating strokes of different gray levels. 

21. The apparatus according to claim 20, wherein 

said stroke separation device judges whether 
to perform a stroke separation using one of an 
inter-class dispersion and a dispersion ratio 
between different strokes. 

22. The apparatus according to claim 4, wherein 

said local binarization device judges which 
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of the background and the stroke a pixel, which is 
judged to be the background pixel by said background 
judgment device, belongs to based on a gray level of 
the pixel. 

5 

23. An image processing method, comprising: 

judging for each target pixel whether a 
target pixel is a background pixel on receipt of a 
multilevel image; and 
10 locally binarizing the target pixel, judging 

which of a background and a stroke the target pixel 
belongs to and outputting a binary image if it is judged 
that the target pixel is not the background pixel. 

15 24. The method according to claim 23, 
further comprising: 

processing the obtained binary image based on 
a ratio of black pixels in a shape-fixed line 
element mask including the target pixel; and 

20 outputting a binary image. 

25. The method according to claim 23, 
further comprising: 

binarizing a partial pattern in a gray scale 
25 image corresponding to a black pixel joint element in 
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the obtained binary image; and 

separating strokes of different gray levels. 

26. A computer-readable storage medium on which 
5 is recorded a program for enabling a computer 

extracting a stroke included in an inputted multilevel 
image to perform a process, said process comprising: 
judging for each target pixel whether a 
target pixel is a background pixel on receipt of a 
10 multilevel image; and 

locally binarizing the target pixel, judging 
which of a background and a stroke the target pixel 
belongs to and outputting a binary image if it is judged 
that the target pixel is not the background pixel. 

15 

27. The storage medium according to claim 26, 
said process further comprising: 

processing the obtained binary image based on 
a ratio of black pixels in a shape-fixed line 
20 element mask including the target pixel; and 
outputting a binary image. 

28. The storage medium according to claim 26, 
said process further comprising: 

25 binarizing a partial pattern in a gray scale 
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image corresponding to a black pixel joint element in 
the obtained binary image; and 

separating strokes of different gray levels. 

29. An image processing apparatus, comprising: 

input means for receiving a multilevel image; and 
background judgment means for judging whether 
a target pixel is a background pixel using a 
standard deviation of gray levels of pixels in a 
vicinity area of the target pixel. 

30. An image processing apparatus, comprising: 

input means for receiving a multilevel image; and 
background judgment means for judging whether 
a target pixel is a background pixel using a gray 
level difference and a standard deviation of gray levels 
of pixels in a vicinity area of the target pixel. 

31. An image processing apparatus, comprising: 

background judgment means for judging for 
each target pixel whether the target pixel is a 
background pixel on receipt of a multilevel image; and 

local binarization means for locally 
binarizing the target pixel, judging which of a 
background and a stroke the target pixel belongs to, 
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and outputting a binary image if it is judged that the 
target pixel is not the background pixel. 

32. A transmission signal transmitting to a 
computer, which extracts a stroke included in an 
inputted multilevel image, a program for enabling the 
computer to perform a process , said process comprising: 

judging for each target pixel whether a 
target pixel is a background pixel on receipt of a 
multilevel image; and 

locally binarizing the target pixel, judging 
which of a background and a stroke the target pixel 
belongs to and outputting a binary image if it is judged 
that the target pixel is not the background pixel. 
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Abstract of the Disclosure 



It is judged for each pixel in an 
inputted multilevel image whether the pixel is a 
background pixel, and the pixel is locally binarized 
if it is judged not to be a background pixel. Then, it 
is judged whether the pixel belongs to a background or a 
stroke, such as of a character, ruled line, etc., and 
a binary image is generated. 
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jeopardize the validity of the application or any patent issued 
thereon. 
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Japanese Language Declaration 



POWER OF ATTORNEY: As a named inventor. I hereby appoint 
th« following attomey(s) and/or agent(s) to prosecute this 
application and transact all business in the Patent and Trademark 
Office connected therewith (Jest nam* and registration mmibar) 



James D. Halsey, Jr., 22,729; Harry John Staas, 22,010; David M. Pitcher, 25,908; John C. Garvey, 28,607; J. Randall Beckers, 30,358; 
William F. Herbert, 31,024; Richard A. Gollhofer, 31,106; Mark J. Henry, 36,162; Gene M. Garner II, 34,172; Michael D. Stein, 37,240; Paul 
1. Kravetz, 35,230; Gerald P. Joyce, III, 37,648; Todd E. Marlette, 35,269; Harlan B. WiJIiams, Jr., 34,756; George N. Stevens, 36,938; 
Michael C. Soldner, P-41,455 and William M. Schertler, 35,348 (agent) 

Sf Sic^T^ Send Correspondence to: 



STAAS & HALSEY 

700 Eleventh Street, N.W. 

Suite 500 

Washington, D.C. 20001 
SfSBSaiSX- • (£ $iiklf$.i2i-&Zr) °"*<* Telephone Calls to: {nam* and talaphona numbar) 



STAAS & HALSEY 
(202) 434-1500 



h£— £j"cii3R — S^^T^ Fu " name of sole or first inventor 

Katsuhito FUJIMOTO 



/■entof s sqnalure ^ , Dale , / 

X^LLT^ ^LUt&r 



Residence 

Kawasaki, Japan 



Citizenship 

Japan 



Post Office Address 

c/o FUJITSU LIMITED, 1 - 1 , Kamikodan ik 



4-chome, Nakahara-ku, Kawasaki-shi 
Kanagawa 211-8588, Japan 



— jt-(S]5S9t^* Full name of second joint inventor, if any 

Atsuko QHftBft 



3R — Jt-*SIS6"W-«- 



V2s/2ot>(? 



ResJdenc* 

Kawasaki, Japan 



Post Office Address 

c/o FUJITSU LIMITED, 1-1, Kamikodanaka 



4-chome, Nakahara-ku, Kawasaki-shi, 
Kanagawa 211-8588, Japan 



(3?Ey»<^|5J3& ( H-frKot,>T tl3IStf-3£ifiL. (Suppry similar information and signature for third and subsequent 

l> ; v ) joint inventors.) 
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Full name of third joint inventor, if any 
Satoshi SAOl 




fcl 


Th i r d |i n v en t o r sosignature Date 






Res i dence 
Kawasaki , Japan 


H fi Citizenship 

Japan 






Post Office Address 
c/o FUJITSU LIMITED, 1-1, Karaikodanaka 


4-chome, Nakahara-ku f Kawasaki- shi, 






Full name of fourth joint inventor, if any 






Fourth inventor's signature Date 


£ fJx 




Residence 


g if Citizenship 






Post Office Address 










Full name of fifth joint inventor, if any 




Btt 


Fifth inventor's signature Date 


{£ pJt 




Resi dence 


H f§ Citizenship 






Post Office Address 








Full name of sixth joint inventor, if any 




Btt 


Sixth inventor's signature Date 


& m 




Residence 


m m 




Citizenship 






Post Office Address 






- i) 


(Supply similar information and signature for 
seventh and subsequent joint inventors. ) 
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